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. I Bookmarks and Watchpoints for Selection and Presentation of Media Streams 

2 

3 BACKGROUND OF THE INVENTION 

4 

5 1 . Field of the Invention 
6 

7 The invention relates to selection and presentation of media streams; in 

8 aspects thereof, the invention relates to bookmarks and watchpoints, and to event han- 

9 dling, for selection and display of videos from media, such as for example DVD and 

10 other related media. 
11 

12 2. Related Art 
I? 

14 In systems for presenting information, such as for example information 

15 relating to audiovisual media, users are able to present information already recorded on 

16 digital media. For example, DVD player systems provide users with the ability to play- 

17 back digital content pre-recorded on DVD media. In the incorporated disclosure (as de- 

18 scribed below) systems provide users with the ability to present media streams either 

19 after, or concurrently, with, download of digital content representing those media 

20 streams from one or more content servers. 
21 
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1 A first problem with known DVD media playback is that the process of 

2 playback involves interaction between the user and a preselected sequence of playback 

3 capabilities, recorded onto the DVD media by someone other than the user (such as a 

4 creator or distributor of that DVD media). For example, playback might involve navi- 

5 gation of a selection menu, might involve playback of audiovisual messages the creator 

6 or distributor has determined are advantageous for the user to be forced to watch (such 

7 as for example an anti-piracy warning, or such as for example one or more advertise- 

8 ments or pre-release announcements). Accordingly, it would at some times be advan- 

9 tageous to avoid being restricted by choices imposed on them by the creator or dis- 

10 tributor of the DVD media. 
H 

12 A second problem with known playback techniques is that the user might 

13 have a substantial library of media streams from which the user might desire to present. 

14 For a first example, the user might wish to present short clips (or "trailers") of individ- 

15 ual movies as part of a process of selection or review of those individual movies. For 

16 example, the user might wish to present a trailer for an individual movie as part of de- 

17 ciding whether to purchase that movie (including for example on-line purchase of a 

18 downloaded copy) or to present that movie to a selected audience (including for exam- 

19 pie determining if that audience was interested in the movie in response to the selected 

20 trailer). For a second example, the user might wish to point to selections from media 

21 streams when analyzing, commenting on, reviewing, or selecting portions thereof. A 

22 first example thereof might be a biography of a known actor or director, including se- 
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1 lections of short clips of scenes used for illustrative and literary purposes. A second ex- 

2 ample thereof might be that the user wishes to present a short clip, such as a favorite 

3 scene, for presentation to a selected audience (including for example just the user them- 

4 self), or as part of a pedagogical exercise (including for example as a demonstration of 

5 acting technique or camera work). 
6 

7 Both these problems are exacerbated by increases in the number of DVD 

8 media providers (whether creators or distributors), and by increases in the number of 
? media streams available to the user. Moreover, both these problems might inhibit de-r 

10 velopment of a convenient III (user interface) for selection and presentation of media 

1 1 streams, whether from recorded physical media or downloaded from one or more con- 

12 tent servers. : 
13 

14 Accordingly, it would be advantageous to provide an improved technique 

15 for video selection and presentation, relatively less subject to problems with known 

16 techniques. 

17 

18 SUMMARY OF THE INVENTION 
19 

20 The invention provides a method and system capable of selection and 

21 presentation of media streams, in which presentation is not substantially limited by any 

22 order imposed by the digital content representative of those media streams. 
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2 In one aspect the invention provides a method and system capable of cre- 

3 ating and referencing a bookmark to a location selected within the media stream, not 

4 necessarily pre-selected by the creator or distributor of that media stream, and possibly 

5 dynamically selected by a recipient of digital content representing that media stream. 

6 In addition to referencing the bookmark, the invention provides a user with the capa- 

7 bility of starting or restarting presentation from the selected bookmark. 
8 

.9 In one aspect, the invention provides a method and system capable of cre- 

10 ating a watchpoint at a location selected within the media stream, or a presentation 

1 1 state otherwise associated with presentation of the media stream, and capable of associ- 

12 ating an one or more events with that watchpoint. For one example, the user might 

13 designate a bookmark at the beginning of a selected film clip, a watchpoint with the end 

14 of that same film clip, and an event associated with the watchpoint, which event directs 

15 a presentation device to return to a presentation state it was at before presenting from 

16 the bookmark. In this example, the film clip effectively acts as a media element capable 

17 of being inserted into another, different, media stream (or more generally, into any 
1$ other user interface or media presentation, the original user interface or media preseii- 

19 tation not necessarily including any original media streams of its own), without in- 

20 volving any other digital content associated with the larger media stream that contains 

21 that film clip. 
22 
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1 In one aspect, the invention provides a method and system capable of se- 

2 lecting one of a set of such film clips (bookmark-watchpoint pairs) for presentation, in 

3 response to a user input, a user preference, a playback device configuration, demo- 

4 graphic information about the user or the playback device, or other pertinent informa- 

5 tion. For a first example, a single DVD might have a first version of its movie better 

6 suited for presentation on a regular television screen, and a second version of its movie 

7 better suited for presentation on a widescreen or HDTV television screen. For a second 

8 example, a single DVD might have a first version of its movie better suited for presen- 

9 tation to a mature audience, and a second version of its movie better suited for presen- 

10 tation to a less-mature audience. For a third example, a single DVD might have a set of 

1 1 versions of its movie with selected different plot points or movie endings, for those difc- 

12 ferent audiences or users whp prefer the movie to have a happy ending or otherwise. 

13 For a fourth example, a single DVD might have a shorter (abridged) and a longer (un- 

14 abridged) version of the same movie, selectable by the user. 
15 

16 In one aspect, the invention provides a method and system capable of 

17 testing one or more state variables at such bookmarks or watchpoints, and of taking dif- 

18 ferent actions in response to distinct values of those state variables. In one embodiment, 

19 the invention provides a user with the capability of designing a sequence of actions 

20 (conditional or unconditional), by which selections from media streams might be pre- 

21 sented in response to digital content representing those media streams and in response 

22 to other information. Such other information might include the scope of a license af- 
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1 forded to the user, demographic or other metadata about that user, or interactive activ- 

2 ity with that user (or more than one such user). Such a sequence of actions might be re- 

3 corded, retrieved, transmitted, or received, similar to digital content representing any 

4 other type of media stream. 
5 

6 . In one embodiment, the invention might include bookmarks created or 

7 edited using one or more techniques or by one or more types of creators. For a first ex- 

8 ample, bookmarks might be created in conjunction with the original content, such as for 

9 example by an author, creator, distributor, or synchronizer, of the original content. For 

10 a second example, bookmarks might be created by a party other than in conjunction 

1 1 with the original content, such as for example by a secondary distributor of the content, 

12 a server providing the content to an end-user along with newly-associated metadata, a 

13 translator of the original content, such as a person adding subtitles in a second or an al- 

14 ternative language or "dubbing" in sound in a second or an alternative, or such as an 

15 editor of the movie for a specific purpose, such as for example a purpose suitable for 

16 airliner use, for children, or for export to selected countries. For a third example, 

17 bookmarks might be created by end-user preferences, either explicit or implicit or some 

18 combination or conjunction thereof, such as for example in response to the end-user 

19 designating a favorite scene, or such as for example in response to the end-user skip- 

20 ping over a disfavored scene, or such as for example in response to demographic infor- 

21 mation about the end-user. Similar to bookmarks, the invention might also include 
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1 watchpoints created using one or more techniques or by one or more types of creators, 

2 as described above with regard to creation or editing of bookmarks. 

3 

4 After reading this application, those skilled in the art would recognize that 

5 the invention is an enabling technology by which substantial advance is.made in the art 

6 of media streams and digital content representative thereof, For example, the invention 

7 might be used to provide one or more of, or some combination or extension or mixture 

8 of, any of the following: 



9 

10 • Saving selected scenes from movies and requesting playback of those scenes, ei- 

U ther in response to user command, pointers to those scenes, or metadata de- 

12 scribing those scenes. 

13 

14 • Pausing movies during presentation oh a first device and requesting playback on 

15 a second device, such as for example in response to user command. 
16 

17 • Assembling a collection of such selected scenes, possibly along with commen- 

18 tary, or possibly embedded in another media stream for which those selected 

19 scenes are exemplary or illustrative. ■ 
20 
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. I • Assembling a collection of bookmarks or watchpoints, possibly along with com- 

2 mentary; maintaining that collection in a database or other metadata; or sending 

3 that collection from one device to another. 
4 

5 • Creating and presenting a demonstration of a home video system, including 

6 demonstration of selection and presentation of media streams as part of the 

7 demonstration, In one embodiment, the demonstration itself might be interac- 

8 tive, and include the capability for the user to select which media streams are se- 

9 lected or presented as part of the demonstration itself. 

10 ; 

11 • Creating and presenting a media stream by which a pseudorandom or random 

12 process determines presentation of such selected scenes, possibly in response to 

13 other information. In one embodiment, the pseudorandom or random process 

14 might select film clips in response to known preferences of the user, thus pro- 

15 viding the capability of a film clip ''screen saver" or a variant of the MTV chan- 

16 nel. 

17 

18 • Pausing, restarting, arid repeating presentation of media streams, even for jnedia 

19 for which such functions are not originally supported. 

20 . 
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1 • Direct access to feature presentations within recorded digital media, without any 

2 substantial requirement for navigating via pre-recorded menus or pre-recorded 

3 advertising, announcements, or warnings. 
4 

5 • Direct access to supplements to such feature presentations, or trailers associated 

6 with such feature presentations, within recorded digital media, without any sub- 

7 stantial requirement for navigating via pre-recorded menus or pre-recorded ad- 

8 vertising, announcements, or warnings. 
9 

10 • Presentation of media streams repre$ented by digital content recorded on multi- 

1 1 pie volumes of digital media, such as for example movies recorded on more than 

12 . one DVD, without any substantial requirement for the user to intervene at a 

13 switching point between those multiple volumes. 
H 

15 • Performing operations external to presentation in response to the media stream, 

16 such as for example turning up room lights at the start of movie credits, dimin- 

17 ishing sound volume during commercial breaks, or pausing presentation in re- 

18 sponse to aspects of the movie (such as for example at a commercial break in a 

19 broadcast movie). 

20 * 

21 • Creating and presenting a computer game using individual film clips, either 

22 from pre-existing DVD media or newly-created digital content. For example, the 
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1 computer game might be interactive, or might respond to external information, 

2 such as for example a television news feed. 
3 

4 In one embodiment, the invention includes a representation of a presenta- 



5 Hon state (herein sometimes called "P state") and a virtual machine state (herein some- 

6 times called '*VM state"), each of which is updated concurrently with presentation of 

7 the media stream. 
8 

9 A bookmark preferably includes a defined P state and VM state to which a 

10 presenting device might be set, with the effect of providing that presenting device with 

1 1 the capability of presentation beginning at that bookmark. In one embodiment, pres- 

12 entation might instead begin at a closest "entry location" designated by the digital con- 

13 tent representing that media stream. In one embodiment, the defined P state or VM 

14 state might include a known "title number" within DVD media on which is recorded 
.15 the digital content representing that media stream. 

17 A watchpoint preferably includes a defined P state and VM state, along 

18 with one or more associated triggerable actions, with the effect that whenever the pre- 

19 senting device attempts to match its P state and VM state with that watchpoint, a suffi- 

20 cient degree of similarity (possibly for a subset of data associated with each such state) 

21 would cause the watchpoint to be triggered and the associated actions would be per-r 

22 formed. For example, the presenting device might attempt to match its P state and VM 
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1 state with selected watchpoints after each known frame of the media stream, or at the 

2 beginning or end of each subsegment of the media stream designated by the digital 

3 content representing that media stream. 
4 

5 INCORPORATED DISCLOSURE 

6 

7 This application claims priority of the following documents, each of which 

S is hereby incorporated by reference as if fully set forth herein. 

9 

10 • US provisional patent application 60/439,330, filed January 31, 2003, in the 

1 1 name of Shawn NEELY, Lawrence KESTELOOT, Margaret NOVOTNY, Marion 

12 . BUCHENAU, Sheila FOLEY, and Michael O'NEIL, attorney docket number 

13 217.1014.01, titled "User Interface for Video Selection and Display", 
14 

15 • US. patent application 10/418,949, filed April 18, 2003, in the name of Shawn 

16 NEELY, Lawrence KESTELOOT, Margaret NOVOTNY, Marion BUCHENAU, 

17 Sheila FOLEY, and Michael O'NEIL, attorney docket number 217.1017.01, titled 

1 8 "Guide-Like User Interface for Video Selection and Display". 
19 

20 • U.S. patent application 10/418,739, filed April 18, 2003, in the name of Shawn 

21 NEELY, Lawrence KESTELOOT, Margaret NOVOTNY, Marion BUCHENAU, 
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1 Sheila FOLEY; and Michael O'NEIL, attorney docket number 217.1018.01, titled 

2 "Mosaic-Like User Interface for Video Selection and Display". 
3 

4 These documents are hereby incorporated by reference as if fully set forth 

5 herein, and are sometimes referred to herein as the "incorporated disclosure". 
6 

7 Inventions described herein can be used in combination or conjunction 

8 with technology described in the incorporated disclosure. 
9 

10 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

ll 

12 In the description herein, a preferred embodiment of the invention is de- 

13 scribed, including preferred process steps and data structures. Those skilled in the art 

14 would realize, after perusal of this application, that embodiments of the invention 

15 might be implemented using a variety of other techniques not specifically described, 

16 without undue experimentation or further invention, and that such other techniques 

1 7 would be within the scope and spirit of the invention. 
18 

19 Lexicography 
20 

21 The general meaning of each of these following terms is intended to be il- 

22 lustrative and in no way limiting. 
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1 

2 • The phrase "media stream" describes information intended for presentation in a 

3 sequence, such as motion pictures including a sequence of frames or fields, or 

4 such as audio including a sequence of sounds. As used herein, the phrase 

5 "media stream" has a broader meaning than the standard meaning for 

6 "streaming media/' (of sound and pictures that are transmitted continuously 

7 using packets and that start to play before all of the content arrives). Rather, as 

8 described herein, there is no particular requirement that "media streams" must 

9 be delivered continuously. Also as described herein, media streams can refer to 

10 other information for presentation, such as for example animation or sound, as 

1 1 well as to still media, such as for example pictures or illustrations, and also to 

12 databases and other collections of information. 
13 

14 • The phrase "digital content" describes data in a digital format, intended to repre- 

15 sent media streams or other information for presentation to an end viewer. 

16 "Digital content" is distinguished from packaging information, such as for ex- 

17 ample message header information. For the two phrases "digital content" and 

18 "media stream," the former describes a selected encoding of the latter, while the 

1 9 latter describes a result of presenting any encoding thereof. 
20 

21 • The phrases "digital media," "DVD media," and the like, describe media capable 

22 of maintaining digital content in an accessible form. DVD media specifically de- 
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1 scribes a family of encoding formats including DVD-video (such as for example 

2 used with media including digital video disks or equivalents thereof). 

3 

4 • The. terms " playback/' "presentation/' and the like, describe presentation (such 

5 as for example to one or more users) of audio, visual, or other features of media 

6 streams. 

7 

8 • The phrase "content server" describes a device (or a portion thereof, or a set of 

9 such devices or portions thereof) capable of sending digital content to recipients: 

10 For example, a content server might include a web server at which a user is pro- 

1 1 vided the capability of purchasing digital media for download. In the context of 

12 this application, there is no particular requirement that the server be (logically or 

13 physically) located tit any particular address or place, or have any particular ar- 

14 chitecture, or use any particular protocol for communication. For example, the 

15 content server might include a process logically available to a local presentation 

16 device. 
17 

18 • The term "bookmark" describes a reference to a logical location selected within a 

19 media stream. In one embodiment, bookmarks are not necessarily pre-selected 

20 by the creator or distributor of that media stream, and are possibly dynamically 

21 selected by a recipient of digital content representing that media stream. In one 
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1 embodiment presentation devices are capable of starting or restarting presenta- 

2 tion from a selected bookmark. 
3 

4 • The term "watchpoint" describes a reference to a logical state of a presentation 

5 . device, such as for example a logical location selected within a media stream. In 

6 one.embodiment, watchpoints are capable of associating an one or more events 

7 therewith, and (preferably) those one or more events might be conditioned on 

8 some other data or state information. For one example, the user might designate 

9 a bookmark at the beginning of a selected film clip, a watchpoint with the end of 

10 that same film clip, and an event associated with the watchpoint, which event di- 

11 rects a presentation device to return to a presentation state it was at before pre- 

12 : senting from the bookmark. In this example, the film clip effectively acts as a 

13 media element capable of being inserted into another, different, media stream, 

14 without involving any other digital content associated with the larger media 

15 stream that contains that film clip. 
16 

17 The scope and spirit of the invention is not limited to any of these defini- 



18 Hons, or to specific examples mentioned therein, but is intended to include the most 

1 9 general concepts embodied by these and other terms. 
20 
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I Alternative Embodiments 
2 

3 Although preferred embodiments are disclosed herein, many variations 

4 are possible which remain within the concept scope, and spirit of the invention. These 

5 variations would become clear to those skilled in the art after perusal of this applica- 

6 tion. 



7 

8 • The invention is not restricted to movies, but is also applicable to pther media 

' .... . 

9 streams, such as for example animation or sound, as well as to still media, such 

10 as for example pictures or illustrations, and to databases and other collections of 

11 information. 

12 

13 Those skilled in the art will recognize, after perusal of this application, 

14 that these alternative embodiments are illustrative and in no way limiting. 
15 

16 TECHNICAL APPENDIX 

17 

18 Other and further information about the invention, and embodiments 

19 thereof, are included in a technical appendix, hereby incorporated by reference as if 

20 fully set forth herein. The technical appendix includes the following documents: 
2i 
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1 • 1 "Automatic Masking." (Paul Rechsteiner, Nik Gervae, Shawn Neely, Michael 

2 Malcolm, Ray DcPaul, Daniel Collens) 

3 • 2 "Deletion of Movies/' (Daniel Collens, Kevin Hui, Lawrence Kesteloot, Richard 

4 Czegledi, Michael Malcolm, Paul Rechsteiner) 

5 • 3 "DVD Import and Cache Misses " (Daniel Collens, Shawn Neely, Richard 

6 Czegledi, Robert Zarnke, Paul Rechsteiner, Kevin Hui) 

7 • 4 "DVD Reader User Experience." (Daniel Collens, Shawn Neely, Lawrence Ke- 

8 steloot) 

9 • 5 "DVD Bookmarks." (Paul Rechsteiner, Shawn Neely, Michael Malcolm, Law- 

10 rence Kesteloot, Daniel Collens, Robert Zarnke) 

11 • 6 "Installation Guide." (Shawn Neely, Lawrence Kesteloot, Daniel Cpllen$, Ray 

12 DePaul, Paul Rechsteiner) 

13 • 7 "OSD Standby Mode." (Paul Rechsteiner, Lawrence Kesteloot, Daniel Collens) 

14 • 8 "Packages, Selections, and Play Items." (Shawn Neely, Lawrence Kesteloot, 

15 Daniel Collens) 

16 • 9 "PVL UI Feature Requirements." (Shawn Neely, Paul Rechsteiner, Robert 

17 Zarnke, Daniel Collens, Richard Czegledi, Lawrence Kesteloot) 

18 • 10 " Kaleidescape Home Video System Functional Overview/' (Paul Rechsteiner, 

19 Daniel Collens, Shawn Neely) 

20 • 11 "Serial Control Protocol." (Lawrence Kesteloot, Richard Czegledi, Paul 

21 Rechsteiner, Michael Malcolm, Daniel Collens) 

22 
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• 12 "Suggested Touch Panel Layouts." (Shawn Neely, Lawrence Kesteloot, Daniel 
Collens) 

• 13 "Ul States." (Shawn Neely, Lawrence Kesteloot, Michael Malcolm, Daniel 
Collens) 

• 14 "User-Defined Favorite Scenes." (Shawn Neely, Lawrence Kesteloot, Michael 
Malcolm, Daniel Collens) 

Inventions described herein can be used in combination or conjunction 
with technology described in the technical appendix. 
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! CLAIMS 
2 

3 1. . A method, including steps of 

4 sending a media stream to a first recipient; 

5 creating a bookmark into that media stream at that first recipient, the 

6 bookmark being nQt pre-$elected by a the creator or distributor of that media stream; 

7 sending that media stream and that bookmark to a second recipient; 

8 dynamically selecting that bookmark at that second recipient; and 

9 presenting that media stream beginning at a point designated by that 

10 bookmark. 
II 

12 2. A method, including steps of 

13 creating a watchpoint describing either (a) a presentation state in a media 

14 stream, or (b) another state associated with presentation of that media stream; and 

15 associating one or more events with that watchpoint; 

16 whereby when said presentation state or other state is reached, those one 

17 or more events are performed with regard to that media stream. 
18 

19 3. A method as in claim 2, 

20 wherein one of those events includes directing a presentation device to go 

21 to a bookmark indicating a different presentation state associated with a banning of a 

22 film clip from that media stream; 

Express Mailing EL 768 962 054 US . 20 



217.1019.Q1 



1 whereby the combination of the bookmark and the watchpoint enable that 

2 film clip to be presented again. 
3 

4 4. A method as in claim 1, including steps of 

5 testing one or more state variables at a selected watchpoint; and 

$ performing ah action associated with one of the events associated with 

7 that watchpoint, in response to a result of the steps of testing. 

8 

9 5. A method as in claim 4, wherein that action includes a sequence of 

10 conditional or unconditional instructions. 

12 . 6. A method as in claim 4, wherein that action includes a sequence Qf 

13 instructions capable of being recorded, retrieved, sent, or received. 

14 ; 

15 7. A method as in claim 4, wherein that action includes presenting 

16 one or more selections from designated media streams, in response to information other 

17 than found in digital content representing those designated media streams. 

18 . 

19 8. A method as in claim 7, wherein that information includes at least 

20 one of: a license afforded to the user, a set of demographic or other metadata about that 

21 user, or a set of information gleaned from interactive activity with that user or more 

22 than one such user. 
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2 9. A method as in claim 2, 

3 wherein one of those events includes directing a presentation device to go 

4 to a bookmark indicating a different presentation state associated with presentation of 

5 that media stream; 

6 whereby the combination of the bookmark and the watchpoint enable a 

7 portion of that media stream to be presented as an film clip capable of being inserted 

8 into a distinct user presentation. 

9 ■ * . 

10 10. A method as in claim 9, including steps of presenting one of a set of 

1 1 such film clips in response to at least one of: a configuration of a presentation device, a 

12 user input, a user preference, a set of demographic information about that user or that 

13 presentation device. 
14 

15 11. A method as in claim 9, wherein that distinct user presentation in- 

16 eludes at least one of: a media presentation, a media stream, a user interface, a user 

17 presentation received from a source device, a user presentation recorded on physical 

18 media. 

19 " 

20 12. A method as in claim 9, including steps of selecting one of a set of 

2 1 media streams each designated by at least one such bookmark. 
22 
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•l 13. A method as in claim 12, wherein a single physical media includes 

2 a first media stream suited for presentation on a first presentation device, and a second 

3 media stream suited for presentation on a second presentation device. 

5 14. A method as in claim 12, wherein a single physical media includes 

6 a first media stream suited for presentation to a set of users having a first demographic, 



7 and a second media stream suited for presentation to a set of users having a second 

8 demographic. 

10 . 15. A method as in claim 12, wherein a single physical media includes 

11 a plurality of media streams each identifiable or selectable in response to at least one 

12 such bookmark. 
13 

14 16. A method as in claim 12, 

15 wherein a single physical media includes a set of media streams differing 

16 in length, and 

n including steps of selecting one such media stream for presentation. 

18 

19 17. A method as in claim 12, 

20 wherein a single physical media includes a set of media streams differing 

21 in storyline, and 

22 including steps of selecting one such media stream for presentation. 



Express Mailing EL 768 962 054 US 23 



7 



217.1019.01 

1 Bookmarks and Watchpoints for Video Selection and Presentation 

2 Paul RECHSTEINER ct al. 

.3 

4 ABSTRACT OF THE DISCLOSURE 

5 

6 Selection and presentation of media streams, in which presentation is not 

7 limited by order imposed by the digital content representative of those media streams, 

8 in response to bookmarks and watchpoints settable and usable by a user. Bookmarks 

9 reference a location within the media stream, not necessarily pre-selected and possibly 

10 dynamically selected, in which a presentation can be started or restarted therefrom. 

1 1 Watchpoints reference a location within the media stream, or a presentation state for the 

12 media stream, in which an event can be triggered thereby. Bookmarks and watchpoints 

13 provide a general technique for presenting selections from media streams in response to 

14 metadata. 
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